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Abstract 

This study reports positive impacts of a modified version of the teacher work sample on 
performance of struggling K-12 readers and on the dispositions of the interns who worked with 
them. The participants included 55 university students in a yearlong, senior-level internship and 
55 elementary students identified as struggling readers. The intervention occurred over 12 weeks 
as part of a reading methods class. The interventions were designed by the interns after using a 
variety of classroom reading assessments to diagnose the areas in which the struggling readers 
needed help. Struggling reader progress was measured by comparing posttest to pretest scores. 
Because of the variety of assessment instruments used, only samples of raw data are reported for 
struggling readers. Interns completed a questionnaire measuring dispositions toward teaching 
struggling readers before and after the intervention. Disposition data were analyzed using a 
matched pair t test. Significant changes in dispositions (p < .05) were reported. 

Introduction 

Documenting student learning is not new to education; however, in the current context of 
accountability this documentation has taken on new dimensions. Now policy makers and the 
general public demand accountability by requiring tests that allege to be able to compare student 
to student, school to school, district to district, state to state and even country to country. Idaho, 
like most other states, established performance standards for K-12 students that will be assessed 
by high stakes exams. The state also passed a reading initiative with requirements for both pre- 
service and practicing teachers. All elementary, pre-service teacher candidates must pass the 
Idaho Comprehensive Literacy Assessment on principles and practices of teaching reading 
before they can be licensed (certified). All teacher candidates in Idaho must also pass technology 
and content exams. The current standards of the National Council for Accreditation of Teacher 
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Education (2001) require institutions to use data to document that candidates have the 
dispositions, knowledge, and skills to help all students learn. This documentation is to be 
organized in an assessment system that can be used for both external review and internal 
program improvement. The two components of NCATE Standard I that institutions find most 
challenging to document are candidate dispositions and K-12 student learning. 

Of overriding concern for teacher educators and K-12 practitioners is that within this 
environment, rife with testing, methods of assessment must be found that contribute to, rather 
than detract from, student learning. Also, it is essential that assessments measure higher levels of 
learning, not only what is easily measured. The teacher work sample, properly applied, can be 
instrumental in addressing these challenges. 

This paper describes the use of a modified version of the teacher work sample approach 
to documenting K-6 student learning and changes in candidates’ dispositions. During the 
internship year candidates diagnosed areas of need for a K-6 struggling reader, prescribed 
intervention lessons and activities, tutored the struggling readers, and documented progress. 
Candidate dispositions were assessed using a questionnaire before and after the tutoring 
experience. 

Related Literature 

A number of researchers found teacher dispositions that were impediments to literacy 
learning by struggling readers (e.g., Allington & Cunningham, 1996; Walmsley & Allington, 
1995). Walmsley and Allington (1995) found a tendency on the part of classroom teachers to 
excuse themselves from the responsibility for educating low-achieving students. Believing that 
specialists and separate programs are the solution, there is often a reduced professional 
accountability by classroom teachers for the progress of struggling readers. Allington and 
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Cunningham (1996) discovered that the presence of special programs often contributed to 
teachers’ belief they were ill-equipped to teach at-risk readers. Moreover, many teachers did not 
believe it was possible for a struggling reader to catch up (Allington, 1995). 

Approaches to changing dispositions seem to be related to research on changes in 
attitudes. Zimbardo and Leippe (1991), when discussing the definition of attitude, stated, “. . .the 
components are not independent or isolated in different comers of the mind. To the contrary 
they can be highly interrelated” (p. 32). They went on to explain that outward behaviors, though 
not the only approach to changing attitudes, can be a powerful attitude change agent. Active 
involvement seems more powerful than passively receiving information. Chan and Cheng 
(2001) reported that knowledge and direct experience had a positive impact on the attitude of 
nurses toward specific groups of patients. This study attempted to document that attitude and 
content knowledge are the responsibility of the preparation program; traditional assessment 
methods normally address only content. David Palmer (2001) collected qualitative data 
documenting factors involved in changing attitudes of elementary candidates toward teaching 
science. Candidates with negative attitudes toward teaching science became positive about 
teaching science after taking a one-semester science methods course. Multiple factors were 
identified as contributing to the change, but no single factor stood out as more important than 
others. Each of the candidates identified somewhat unique factors as important to their attitude 
change. 

Documenting the impact of teacher education candidates on K-12 students’ learning is 
one way to demonstrate program effectiveness. The Teacher Work Sample Methodology, 
developed over the past decade at the University of Western Oregon (UWO), can provide this 
kind evidence. 
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Elements of the methodology are as follows: 

■ Sample of work 

■ Targets for learning 

■ Measures of learning 

■ Descriptors of process 

■ Descriptors of context 

■ Analyses of learning gains 

■ Reflection and next steps 

(Schalock, Schalock, McConney, Brodsky, & Myton, 2002, pp. 3-4) 

The process can be adapted to fit a variety of learning goals and contexts. While this 
performance assessment is not intended to be the sole indicator of candidate proficiency and 
program quality, it does provide evidence that the candidate can facilitate learmng with K-6 
students. In addition, it represents an attempt to improve instruction and learning through the use 
of assessment. 

The studies described above provide suggested ways to address the need to document 
program quality by assessing K-12 student learning and approaches to changing candidate 
dispositions related to teaching. The objectives of this study are as follows; 

• This study will attempt to document the efficacy of reading methods courses as measured 

by assessing learning of struggling readers who have been tutored by elementary 

education candidates. 

• This study will attempt to document changes in candidate dispositions toward teaching 

struggling readers. 
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Reading Program Description 

Elementary education majors at Northwest Nazarene University are required to take 
three reading courses. The content of the three courses includes the requirements of the Idaho 
Reading Initiative. During their junior year, elementary education candidates take Fundamentals 
of Reading and Teaching Reading in the Content Areas. In the third reading course, At-Risk 
Readers, senior level candidates are equipped with the ability to diagnose literacy needs and 
accelerate the progress of struggling readers. This course is taken in a condensed time frame in 
the fall semester of an integrated yearlong internship. The internship provides opportunity for 
putting reading theory into practice. Candidates work one-to-one with a weak reader in the 
assigned Professional Development School (PDS) classrooms. Before the reading coursework is 
completed, each candidate meets with the PDS teacher to identify a struggling reader in need of 
intensive literacy intervention. Through a teacher work sample process, candidates document 
their ability to diagnose needs, plan instruction, deliver instruction, and assess progress of a 
struggling reader. 

Method 

Subjects 

The population for this study included 55 senior-level teacher candidates and 55 K-6 
students identified by their classroom teachers as struggling readers. The teachers were 
elementary education majors at Northwest Nazarene University, a small private liberal arts 
university located in southwest Idaho. The K-6 students attended public schools that served as 
PDS sites for the University. These schools serve populations ranging from a largely 
homogenous, middle-class, suburban to diverse, lower socio-economic groups. 
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Design and Procedure 

During 2000 and 2001 elementary education candidates were given a ten-item questionnaire (see 
Appendix for instrument) at the beginning of the first semester of the yearlong internship to 
assess the following dispositions: 

• Belief that classroom teacher is responsible to teach struggling readers 

• Belief in the efficacy of classroom teachers to teach struggling readers 

• Self-efficacy to teach struggling readers 

• Responsiveness and persistence in trying to meet the needs of all learners 

• Belief that all students can learn. 

The responses were recorded on a four point Likert scale ranging from strongly agree to 
strongly disagree. For some items the desired response was strongly agree, for others the 
desired response was strongly disagree. The same instrument was administered at the end of the 
study. The most desired responses were was coded as four, second most desired responses were 
coded as three, third most desired response was coded as two and the least desired response was 
coded as one. A paired-sample t test was applied to the results (p< .05). 

The intervention was based on a modified version of the UWO Teacher Work Sample 
model. A key difference was that this process provided evidence of the growth of one student 
rather than a group of students. Components of these teacher work samples compiled over the 
course of the semester included: 

• Learning-Teaching Context 

• Pre- Assessment Profile 

• Intervention Plans 

• Formative Assessment Notes 




8 



Running Head: Teacher Work Samples: Struggling Reader Performance & Candidate Dispositions 



8 



• Post-Assessment Profile 

• Analysis of Student Learning 

• Evaluation and Reflection 

Learning-Teaching Context. Prior to working with the K-6 student, the candidate 
consulted with the PDS teacher and reviewed school records to gain a preliminary understanding 
of the student. In this section of the teacher work sample, the candidates described the 
characteristics of the student for whom he/she would provide literacy support. The following 
information was included: 

• Student's name 

• Student's gender 

• School and grade level 

• Classroom teacher 

• Description of student's participation in an Extended Year Reading Program or 
other reading intervention program during school year, if applicable 

• Description of other services provided now or in previous years to support 
learning 

• Previous Idaho Reading Indicator (IRI) scores and dates (if available) 

• Anecdotal information from classroom teacher about student's reading difficulty 

• Candidate's observations of student characteristics prior to intervention 

• Classroom characteristics (classroom environment, classroom routines, 
motivational strategies used by teacher, grouping patterns, other factors within 
classroom that may impact this student's learning) 
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Pre-Assessment Profile. During the reading coursework, candidates became skilled in 
the use of a variety of reading assessments for pinpointing learning needs in phonological 
awareness, phonics, sight word recognition, fluency, morphological analysis, syllabication, 
vocabulary, and comprehension. In order to determine the student s literacy strengths and needs, 
each candidate administered a series of developmentally appropriate pre-assessments most of 
which were from the Consortium of Reading Excellence (CORE) Assessing Reading: Multiple 
Measures from Kindergarten through Eighth Grade (1999). The candidate recorded dates and 
scores on the Pre-Assessment Profile. The candidate, the PDS teacher, and the course instructor 
analyzed performance on these pre-assessments. Two learning goals were identified based on 
the student’s performance. 

Intervention Plans. The teacher work sample included a minimum of 12 lesson plans 
geared toward the identified learning goals. Each intervention session was approximately 20-30 
minutes in length. Candidates averaged two sessions per week with their struggling readers. 

Each intervention plan included the following: 

• Goal(s) being worked on in this lesson 

• Information gathered in previous work with the student that determined the need 
for this lesson 

• Instructional strategies, word lists, and plans 

• Reflections about successes, challenges, and possible needs in future interventions 
Formative Assessment Notes. During the reading coursework, candidates learned about 

the importance of monitoring growth throughout the instructional sequence through informal 
measures of student progress. As they worked with struggling readers, they used informal 
assessments and anecdotal notes to document progress toward meeting the two learning goals. 
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An ongoing list of intervention activities and results of formative assessments that related to each 
learning goal was documented as shown in Table 1. 

Table 1 



Sample of Formative Assessment Notes 



learning goal 


Intervention activities 


Formative assessment notes 


Improve sight word 
recognition. 


1. Dolch Bingo List 1 

2. Fishing for Dolch words List 1 

3. Spell-Out Strategy List 1 

4. I Spy List 1 — Word Wall 

5. 

6. 

7. 

8. 


1 . 11-15 Dolch sight word 
check - List 1 22/40 

2. 1 1-22 Dolch sight word 
check - List 1 27/40 (still 
having much trouble with 
where, away, three) 



Post-Assessment Profile. Upon completion of the 12 intervention lessons over a three- 
month period, each candidate administered post-assessments to measure progress in the two goal 
areas. This provided a summative measure of student performance in the targeted areas. 

Analysis of Student Learning. In this section of the teacher work sample, each candidate 
provided data to demonstrate student learning that occurred during the intervention sequence. 
After post-assessments were administered and scored, the candidate listed the two learning goals 
and created a chart to show assessments used and scores and dates for pre-assessments and post- 
assessments for each learning goal. Using percentages, where possible, the candidate showed 
learning gains made for each goal. The candidate then made a professional judgment about 
whether the learning goal was met, partially met, or not met. 

Evaluation and Reflection. For the final section of the teacher work sample, candidates 
wrote a reflective essay in which they evaluated the effectiveness of their teaching practice and 
its impact on student learning. They wrote to the following prompts: 

• To what extent were the learning goals for your intervention sequence met? 

Provide evidence to support your response. 
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• What questions or issues does this intervention sequence reveal about your 
teaching or the student with whom you worked? 

• What aspects of your intervention sequence were especially successful or 
effective? Why? 

• How might you teach this intervention sequence differently if you were to do it 
again? Why? 

• What are your recommendations for this student's next steps in literacy 
instruction? 

• Summary of PDS teacher's professional judgment of student's progress in literacy. 
Presentation to PDS Faculty 

As a follow-up during the second semester, candidates continued to reflect on what they 
learned about meeting the needs of struggling readers. Working with the faculty member who 
had guided them through the teacher work sample process, they compared the K-6 work sample 
data by grade level to show the range of gains for each literacy goal for all of the tutored 
students. Each candidate identified a research question about working with struggling readers. 
Books and educational journals were consulted in order to find what research and best practice 
indicates about that question. Each PDS-based cohort of candidates presented their data and 
review of literature regarding their research questions to the faculty their PDS site. 

Results 

The impact of the interventions on student learning was positive for all K-6 subjects m almost all 
of the literacy goals areas that were addressed by the candidates. Because of the variety of 
assessment instruments used and the variety of grade levels involved it was not appropriate to 
apply inferential statistics to the gains. The gains were reported as raw data with percentage 
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gains. A sample of the data is listed in Table 2. The largest gains were made in sight word 
recognition, phonological awareness and phonics. The smallest gains were made in 
comprehension. 

In addition to analyzing quantitative data, each candidate met with his/her PDS teacher at 
the conclusion of the intervention to discuss their informal observations of the subject’s literacy 
progress. A frequent indicator of growth cited in the candidates’ Evaluation and Reflection 
papers was the child’s increase in confidence, as indicated by his/her eagerness to participate in 
reading activities with the class. Suffice to say the gains were important to the students in the 
sample. 

Table 2 



Examples of Analyses of Student Learning 



Learning Goal 


Pre 


Date 


Post 


Date 


Gain 
+ or - 


Learning 

Goal 


Improve Phonological 
Awareness 

■ Phonological Awareness 
Screening Test 


15/30 = 50% 


9-9-01 


22/30 = 73% 


12-4-01 


+23% 


partially 

met 


Improve ability to recognize 
and blend consonant, short 
vowel, and long vowel sounds. 
■ CORE Phonics Survey 
Subtests 


33/73 = 45% 


9-11-01 


59/73 = 80% 


12-6-01 


+35% 


partially 

met 


Improve Sight Word 
Recognition 
■ Dolch Sight Word 
Recognition Lists 1, 2 


30/92 = 33% 


9-11/01 


81/92 = 88% 


12-6-01 


+55% 


met 



Note. Additional data is available upon request. 

Perhaps the most important finding was the positive impact the tutoring experience had 
on candidate dispositions. Table 3 is from an SPSS output of a paired-sample t test comparing 
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responses on the questionnaire given prior to the tutoring experience with responses on that same 
questionnaire given after the tutoring experience. 

Table 3 

Paired Samples t Test of Items on Candidate Disposition Questionnaire 



Item 

pairs 


Mean paired 
differences 


SD 


ES of 
mean 


95% Confidence 
Interval 


t 


df 


Sig. (2- 
tailed) 










Upper 


Lower 








PREQl - 
POSTQl 


-.064 


.7580 


.1022 


-.269 


.141 


-.623 


54 


.536 


PREQ2- 

POSTQ2 


-.045 


.7470 


.1007 


-.247 


.157 


-.451 


54 


.654 


PREQ3 - 
POSTQ3 


.055 


.8696 


.1173 


-.181 


.290 


.465 


54 


.644 


PREQ4- 

POSTQ4 


-.145 


.5153 


.0695 


-.285 


-.006 


-2.094 


54 


.041* 


PREQ5- 

POSTQ5 


-.164 


.7641 


.1030 


-.370 


.043 


-1.588 


54 


.118 


PREQ6- 

POSTQ6 


-.727 


.7566 


.1020 


-.932 


-.523 


-7.129 


54 


.000* ♦ 


PREQ7- 

POSTQ7 


-.182 


.6963 


.0939 


-.370 


.006 


-1.936 


54 


.058 


PREQ8- 

POSTQ8 


-.227 


.6223 


.0839 


-.395 


-.059 


-2.709 


54 


.009* 


PREQ9- 

POSTQ9 


-.018 


.6524 


.0880 


-.195 


.158 


-.207 


54 


.837 


PREQIO- 

POSTQIO 


.027 


.7228 


.0975 


-.168 


.223 


.280 


54 


.781 



Note. SPSS subtracts the post measure from the pre-measure. Therefore the signs are reversed 
(i.e., an increase in score from pre-assessment to post-assessment appears with a negative sign 
and a decrease from pre to post appears as a positive sign). 

* p < .05. ** p < .001. 

Only two items (item 3, item 10) showed a decline in disposition scores; neither decline 
was significant. Items 4, 6, and 8 showed a significant improvement (p< .05) in candidate 
disposition. 
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Those items are as follows: 

4. It is the responsibility of kindergarten teachers to work with their students who are 
experiencing difficulty with oral language and concepts about print. (Desired 
response toward Strongly Agree) 

6. I am not equipped with instructional strategies for teaching struggling readers to read 
well. (Desired response toward Strongly Disagree) 

8. The top priority of teachers in grades 1-3 should be helping struggling readers learn to 
read well. (Desired response toward Strongly Agree) 

These responses are indicators of the following dispositions: 

Item #4 Belief that classroom teacher is responsible to teach struggling readers. 

Item #6 Self-efficacy to teach struggling readers 

Item #8 Responsiveness and persistence in trying to meet the needs of all learners 

The greatest change was in candidates’ belief in their ability to teach readers who struggle. The 
growing sense of empowerment that candidates experienced is conveyed by the data reported in Table 3; 
however, it is most powerfully expressed by a candidate's reflection. 

As much as I hate to say it, this work with an at-risk reader made me fearful and uncomfortable at 
the beginning. It wasn’t the course or my student that troubled me, but the lack of confidence I had 
in myself about the knowledge of this stuff. But working with Jose has been a highlight for me this 
semester! He showed a passion for learning, which came as a shock to me. From my standpoint, a 
child who is struggling so much would do anything but enjoy trying to learn. The two intervention 
goals for Jose were definite necessities. He worked very hard on sight words, phonological 
awareness, and blending of sounds in reading and writing. The goals for this sequence turned out to 
be very successfully met. Jose’s strongest jump was in his sight word recognition, with a gain of 
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60% ! I could see the evidence of this gain and his 48% gain in blending sounds as I listened to him 
read throughout the intervention sequence. Wlien I first started testing him, he struggled with a pag* 
of six words. Last week he read a story to me that had 1 5-20 words on each page. Now his 
frustration in reading has changed to excitement and interest! I have conquered that which 
intimidated me, and I have realized that every child with even the smallest amount of potential can 
be nurtured and matured. (K. Langley, reflection journal, December 2001) 

Discussion 

Now in the third year of using this teacher work sample process, faculty have identified 
the following benefits. First, it is evident that pre-service teachers can positively impact the 
reading proficiency of struggling K-6 students. Combining coursework with an extended 
practicum during the internship year is a powerful way to equip pre-service teachers with skills 
for diagnosing and meeting literacy needs of readers who are experiencing difficulty. By 
documenting student gains, candidates learn the importance of using data to drive instruction. 

This experience empowers elementary candidates with abilities necessary for effective reading 
instruction in their chosen profession, while providing a much-needed safety net for struggling 
readers in PDS classrooms. By observing where candidates have difficulty in their work with 
struggling readers, faculty can improve the curriculum and instruction in reading methods 
courses. 

Second, and perhaps most important, the teacher work sample experience has a positive 
affect on candidate dispositions. While at the outset of the tutoring experience many candidates 
reported having some of the attitudes and beliefs reported in the literature, their informed and 
focused work with a struggling reader over the course of a semester changed their outlook. The 
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data supports that these candidates are entering the profession believing that they are empowered 
to help all students and learn. 

A third insight of faculty involved with this project is that practicing teachers benefit 
from exposure to current research and practice in reading instruction. At each of the 
presentations made by candidate cohorts to PDS faculties, teachers voiced appreciation for the 
current, research-based information and requested that the university continue to hold these 
follow-up presentations each year. This professional development component of the project 

serves both candidates and educators in the PDS schools. 

In looking ahead, faculty can see the possibility for other research studies related to this 
teacher work sample process. A longitudinal study of K-6 students who were served by this 
program could determine if the gains continue. It is important to know whether this series of 
interventions is the jump-start struggling readers need in order to eventually close the gap with 
their more successful peers or whether their gains will dwindle away without further 
intervention. It would also be helpful to compare gains of students who receive one-to-one 
tutoring with a control group who do not receive tutoring services. What level of training is 
necessary for tutors to be effective in assisting struggling readers? 

The teacher work sample process provides a powerful method of closing the gap for 
struggling readers while documenting the learning of candidates and their students. Considering 
the benefits for all concerned, it is important to find other ways to incorporate this type of 
performance assessment into teacher preparation programs. By demonstrating one aspect of 
candidates’ proficiency through their ability to effect student progress in reading, the institution 
is partially addressing a requirement of the accountability movement in ways that strengthen, not 
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impede, student learning. Documenting changes in candidates’ dispositions is also an important 
accountability measure. 
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Appendix 

Survey of Elementary Preservice Teachers 



ID Number; Date: 

This is an optional, anonymous survey. It is not a requirement of any course and will not 
affect your grades in any way. It is being used in conjunction with a research study being 
conducted by Dr. Karen Blacklock. For each statement, place an X in the box that best 
represents your current beliefs and feelings . Thank you for your helpful information. 





Strongly 

Disagree 


Disagree 


Agree 


Strongly 

Agree 


1. Reading specialists. Title I teachers, and special 

education teachers have more responsibility than regular 
elementary classroom teachers to teach struggling 
readers to read well. 










2. It is the responsibility of teachers in grades 1-3 to teach 
their struggling readers to read well. 










3. It is the responsibility of teachers in grades 4-6 to teach 
their struggling readers how to read well. 










4. It is the responsibility of kindergarten teachers to work 
with their students who are experiencing difficulty with 
oral language and concepts about print. 










5. In general, regular elementary classroom teachers do not 
have the expertise required to teach struggling readers 
to read well. 










6. I am not equipped with instructional strategies for 
teaching struggling readers to read well. 










7. It requires a ir^aster's degree in reading or special 

education to be equipped to teach struggling readers to 
read well. 










8. The top priority of teachers in grades 1-3 should be 
helping struggling readers learn to read well. 










9. One of the top priorities of teachers in grades 4-6 should 
be helping struggling readers learn to read well. 










10. With appropriate instruction it is possible for most 

struggling readers to "catch up" to grade level in reading 
proficiency. 
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